<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="description" content="">
    <meta name="author" content="">
    <meta name="keyword" content="">
    <link rel="shortcut icon" href="/node-mongodb-native/img/favicon.png">

    <title>Metadata</title>

    <link href="/node-mongodb-native/css/bootstrap-theme.css" rel="stylesheet">
    <link href="/node-mongodb-native/assets/font-awesome/css/font-awesome.min.css" rel="stylesheet" />
    <link href="/node-mongodb-native/css/style.css" rel="stylesheet">
    <link href="/node-mongodb-native/css/style-responsive.css" rel="stylesheet" />
    <link href="/node-mongodb-native/css/monokai_sublime.css" rel="stylesheet" />

  </head>

  <body>

  <section id="container" class="">


      <header class="header black-bg">
            <div class="toggle-nav">
                <i class="fa fa-bars"></i>
                <div class="icon-reorder tooltips" data-original-title="Toggle Navigation" data-placement="bottom"></div>
            </div>


            <a href="/node-mongodb-native/" class="logo"><img src="/node-mongodb-native/img/logo-mongodb-header.png" style="height:40px;"></a>

            <div class="nav title-row" id="top_menu">
                <h1 class="nav top-menu"> Metadata </h1>
            </div>
      </header>



<aside>
    <div id="sidebar"  class="nav-collapse ">

        <ul class="sidebar-menu">




            <li class="sub-menu active">
            <a href="javascript:;" class="">
                <i class='fa fa-book'></i>

                <span>schema design</span>
                <span class="menu-arrow fa fa-angle-down"></span>
            </a>
                <ul class="sub open">

                    <li><a href="/node-mongodb-native/schema/chapter1/"> Introduction </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter2/"> Schema Basics </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter3/"> MongoDB Storage </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter4/"> Indexes </a> </li>

                    <li class="active"><a href="/node-mongodb-native/schema/chapter5/"> Metadata </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter6/"> Time Series </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter7/"> Queues </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter8/"> Nested Categories </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter9/"> Account Transactions </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter10/"> Shopping Cart </a> </li>

                    <li><a href="/node-mongodb-native/schema/chapter11/"> Sharding </a> </li>

                </ul>

              </li>


                <li>
                <a class="" href="https://mongodb.github.io/node-mongodb-native/2.2">
                    <i class='fa fa-file-text'></i>

                    <span>2.0 driver docs</span>
                </a>

              </li>


                <li>
                <a class="" href="https://mongodb.github.io/node-mongodb-native/2.2">
                    <i class='fa fa-file-text'></i>

                    <span>1.4 driver docs</span>
                </a>

              </li>


                <li>
                <a class="" href="https://mongodb.github.io/node-mongodb-native/2.2">
                    <i class='fa fa-file-text'></i>

                    <span>Core driver docs</span>
                </a>

              </li>

            <li> <a href="https://groups.google.com/forum/#!forum/learn-mongo-db-the-hardway" target="blank"><i class='fa fa-life-ring'></i>Issues & Help</a> </li>


        </ul>

    </div>
</aside>




      <section id="main-content">
          <section class="wrapper">













              <div class="row">
                  <div class="col-md-1">


                      <a class="navigation prev" href="/schema/chapter4">
                      <i class="fa fa-angle-left"></i>
                      </a>


                  </div>
                <div class="col-md-10">
                    <section class="panel">



                    <div class="panel-body">



<h1 id="metadata:88f6cb0c2dababd96ac958a2c7592492">Metadata</h1>

<p><img src="/images/originals/metadata.png" alt="Metadata, courtesy of http://www.flickr.com/photos/sjcockell/6126442977" />
</p>

<p>Metadata is data that describes and gives information about other data. A classic example is the information about a digital image file such as the ISO settings, resolution, pixel depth, exposure, camera settings, camera type and so forth.</p>

<p>The Metadata Schema is to handle the situation where the information we wish to store in a document has a varying number of fields but we still want to retain the ability to search efficiently on any field in the metadata. Let&rsquo;s use the example of a digital image file.</p>

<h2 id="model:88f6cb0c2dababd96ac958a2c7592492">Model</h2>

<p>Let&rsquo;s look at sample metadata for an image.</p>

<table>
<thead>
<tr>
<th>Field</th>
<th>Value</th>
</tr>
</thead>

<tbody>
<tr>
<td>File name</td>
<td>img_1771.jpg</td>
</tr>

<tr>
<td>File size</td>
<td>32764 Bytes</td>
</tr>

<tr>
<td>MIME type</td>
<td>image/jpeg</td>
</tr>

<tr>
<td>Image size</td>
<td>480 x 360</td>
</tr>

<tr>
<td>Camera make</td>
<td>Canon</td>
</tr>

<tr>
<td>Camera model</td>
<td>Canon PowerShot S40</td>
</tr>

<tr>
<td>Image timestamp</td>
<td>2003-12-14 12-01-44</td>
</tr>

<tr>
<td>Image number</td>
<td>117-1771</td>
</tr>

<tr>
<td>Exposure time</td>
<td><sup>1</sup>&frasl;<sub>500</sub> s</td>
</tr>

<tr>
<td>Aperture</td>
<td>F4.9</td>
</tr>

<tr>
<td>Exposure bias</td>
<td>0 EV</td>
</tr>

<tr>
<td>Flash</td>
<td>No, auto</td>
</tr>
</tbody>
</table>

<p>The first naive approach to modeling this might be to just translate the table directly into a corresponding document.</p>

<pre><code class="language-json">{
  &quot;File name&quot;       : &quot;img_1771.jpg&quot;,
  &quot;File size&quot;       : &quot;32764&quot;,
  &quot;MIME type&quot;       : &quot;image/jpeg&quot;,
  &quot;Image size&quot;      : {&quot;width&quot;: 480, &quot;height&quot;: 360},
  &quot;Camera make&quot;     : &quot;Canon&quot;,
  &quot;Camera model&quot;    : &quot;Canon PowerShot S40&quot;,
  &quot;Image timestamp&quot; : ISODate(&quot;2014-01-01T10:01:00Z&quot;),
  &quot;Image number&quot;    : &quot;117-1771&quot;,
  &quot;Exposure time&quot;   : &quot;1/500 s&quot;,
  &quot;Aperture&quot;        : &quot;F4.9&quot;,
  &quot;Exposure bias&quot;   : &quot;0 EV&quot;,
  &quot;Flash&quot;           : &quot;No, auto&quot;
}
</code></pre>

<p>One of the shortcomings of this model is the need for multiple indexes for multiple fields, increasing the time it takes to write documents and possibly hitting the max number of indexes available on a single collection. For each index you add to a collection it takes longer to insert new documents as all indexes have to potentially be updated.</p>

<p>Another possibility is to leverage the fact that one can index arrays of objects easily. Let&rsquo;s modify the schema above to leverage this.</p>

<pre><code class="language-json">{
  &quot;metadata&quot;: [
    {&quot;key&quot;: &quot;File Name&quot;, &quot;value&quot;: &quot;img_1771.jpg&quot;},
    {&quot;key&quot;: &quot;File size&quot;, &quot;value&quot;: 32764},
    {&quot;key&quot;: &quot;MIME type&quot;, &quot;value&quot;: &quot;image/jpeg&quot;},
    {&quot;key&quot;: &quot;Image size&quot;, &quot;value&quot;: {&quot;width&quot;: 480, &quot;height&quot;: 360}},
    {&quot;key&quot;: &quot;Camera make&quot;, &quot;value&quot;: &quot;Canon&quot;},
    {&quot;key&quot;: &quot;Camera model&quot;, &quot;value&quot;: &quot;Canon PowerShot S40&quot;},
    {&quot;key&quot;: &quot;Image timestamp&quot;, &quot;value&quot;: ISODate(&quot;2014-01-01T10:01:00Z&quot;)},
    {&quot;key&quot;: &quot;Image number&quot;, &quot;value&quot;: &quot;117-1771&quot;},
    {&quot;key&quot;: &quot;Exposure time&quot;, &quot;value&quot;: &quot;1/500 s&quot;},
    {&quot;key&quot;: &quot;Aperture&quot;, &quot;value&quot;: &quot;F4.9&quot;},
    {&quot;key&quot;: &quot;Exposure bias&quot;, &quot;value&quot;: &quot;0 EV&quot;},
    {&quot;key&quot;: &quot;Flash&quot;, &quot;value&quot;: &quot;No, auto&quot;}
  ]
}
</code></pre>

<p>Next lets add a compound index on the metadata array.</p>

<pre><code class="language-json">var col = db.getSisterDB(&quot;supershot&quot;).images;
db.images.ensureIndex({&quot;metadata.key&quot;: 1, &quot;metadata.value&quot;: 1});
</code></pre>

<p>So how do we actually query the data to efficiently use the index. Let&rsquo;s take a look at two query operators called <strong>$all</strong> and <strong>$elemMatch</strong> and how we can leverage them.</p>

<h3 id="all:88f6cb0c2dababd96ac958a2c7592492">$all</h3>

<p>The <strong>$all</strong> operator is defined as selecting all the documents where the value of a field is an array that contains all the specified elements.</p>

<pre><code class="language-js">var col = db.getSisterDB(&quot;supershot&quot;).images;
col.findOne({tags: {$all: [ &quot;appliance&quot;, &quot;school&quot;, &quot;book&quot; ]}});
</code></pre>

<h3 id="elemmatch:88f6cb0c2dababd96ac958a2c7592492">$elemMatch</h3>

<p>The <strong>$elemMatch</strong> operator is defined as matching more than one component within an array element.</p>

<pre><code class="language-js">var col = db.getSisterDB(&quot;supershot&quot;).images;
col.findOne({metadata: {$elemMatch: {key: &quot;File Name&quot;, value: &quot;img_1771.jpg&quot;}}});
</code></pre>

<p>The <strong>$elemMatch</strong> operator looks like the obvious first choice. However the problem is that our <strong>metadata</strong> array is defined as objects that all have <strong>key</strong> and <strong>value</strong>. If you attempt to enter multiple matches using <strong>key</strong> and <strong>value</strong> in the <strong>$elemMatch</strong> only the last pair will be used.</p>

<p>If you wish to locate a photo that has <strong>MIME type</strong> equal to <strong>image/jpeg</strong> and also <strong>Flash</strong> equal to <strong>No, auto</strong> we need to combine <strong>$all</strong> and <strong>$elemMatch</strong>.</p>

<p>Let&rsquo;s take a look at how to correctly pick the document.</p>

<pre><code class="language-js">var col = db.getSisterDB(&quot;supershot&quot;).images;
col.find({ metadata: { $all: [
            { &quot;$elemMatch&quot; : { key : &quot;MIME type&quot;, value: &quot;image/jpeg&quot; } },
            { &quot;$elemMatch&quot; : { key: &quot;Flash&quot;, value: &quot;No, auto&quot; } }
          ]}
       }).toArray();
</code></pre>

<p>The first <strong>$elemMatch</strong> will use the index to locate all the documents with the <strong>MIME type</strong> equal to <strong>image/jpeg</strong> and then filter on the <strong>Flash</strong> key.</p>

<blockquote>
<p><strong>Indexes</strong></p>

<p>One of the thing to remember when it comes to indexes is that the more indexes are on a collection the more <strong>BTree&rsquo;s</strong> need to be updated each time a document is inserted or updated causing additional overhead and IO. In this case we replace 12+ indexes with a single compound index saving both space as well as increasing insert and update performance.</p>
</blockquote>



                <div class="col-md-1">


                    <a class="navigation next" href="/schema/chapter6">
                        <i class="fa fa-angle-right"> </i>
                    </a>


                </div>
              </div>

          </section>
      </section>

  </section>


    <script src="/node-mongodb-native/js/jquery-2.1.1.min.js"></script>
    <script src="/node-mongodb-native/js/jquery.scrollTo.min.js"></script>
    <script src="/node-mongodb-native/js/bootstrap.min.js"></script>

    <script src="/node-mongodb-native/js/highlight.pack.js"></script>
    <script>hljs.initHighlightingOnLoad();</script>
    <script src="/node-mongodb-native/js/scripts.js"></script>
    <script async defer id="github-bjs" src="/node-mongodb-native/js/buttons.js"></script>
    <script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','//www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-41953057-1', 'auto');
  ga('send', 'pageview');

</script>
  </body>
</html>
